The least squares algorithm, parametric system identification and bounded noise

Al~trad-The least squares parametric system identification algorithm is analyzed assuming that the noise is a bounded signal. A bound on the worst-case parameter estimation error is derived. This bound shows that the worst-case parameter estimation error decreases to zero as the bound on the noise is decreased to zero. 1. Introduction THE LEAST SQUARES ALGORITHM, due to Gauss, is one of the most widely used algorithms in science. It has been extensively studied and used for parametric system identification-see, for example, the book by Ljung (1987). It is very well known that the least squares algorithm enjoys certain optimality properties under suitable stochastic assumptions about exogeneous noise. In contrast, some recent papers have taken a worst-case deterministic approach to identification. See the recent paper by Khargonekar (1993) for a discussion of this general area and an extensive list of references. In particular, our work is most closely related to the work on time-domain worst-case identification problems (Chen et al. and the references cited in these papers). Our work has grown out of a need to make connections between the classical identification theory and the more recent work in the area of robust identification. Towards this goal, in this paper we have investigated the performance of the least squares algorithm in the presence of worst-case bounded noise. In the result of this paper, we derive a bound on the worst-case parameter estimation error using the least squares algorithm in the presence of arbitrary bounded noise. This error bound shows that if the input is chosen to be a pseudorandom binary sequence, the worst-case parameter estimation error decreases to zero as the noise bound decreases to zero. In the terminology introduced by Helmicki et al. (1991), the least squares algorithm is robustly convergent. [We note that there is an important technical difference between the notion of robust convergence used in this paper and that in Heimicki et al. (1991).] While the problem formulation is motivated from the deterministic worst-case identification theory, the techniques employed in this paper draw upon the results in classical identification theory (Ljung, 1987). These results are similar in spirit to those in Wahlberg and Ljung (1992).